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(54) Title: METHOD FOR DATA PREPARATION AND WATERMARK INSERTION 



(57) Abstract 

A method for data preparation and watermark 
insertion. The method includes the step of preparing 
the data at a first time by manipulating at least one set 
of the data characteristics (306) for subsequent insertion 
of a first watermark (2). In a preferred embodiment of 
the method of the present invention the method further 
includes the step of inserting the first watermark by 
manipulating the set of data characteristics at a second 
time subsequent to the first time. In a variation of the 
present invention a method for inserting a watermark 
into compressed data is provided. The compressed data 
has sets of data characteristics. The method includes the 
steps of inserting a watermark by manipulating the set 
of data characteristics; and optimizing the manipulated 
data by modifying the compressed data characteristics 
subject to a set of constraints (308, 310). 



300^ 




Derive a signal from a set of 
data characteristics 



Determine the set of all possible 
changes that satisfy the constraints 



308 



Select the change that would optimize 
the similarity between the derived and 
target watermark signals in the 
absence of any future changes. 

. 312 
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METHOD FOR DATA PREPARATION AND WATERMARK INSERTION 



The present invention relates generally to preventing unlawful 
5 copying of audio, video and other media that can be digitized 
and, more particularly, to a method for inserting a watermark 
into digitized data. 

Methods are known to insert a watermark into an image or audio 
10 or video data. One purpose of such a watermark is for copy 
generation management, e.g. allowing a single generation of 
copies to be made from a master, but no subsequent copies to 
be made from the first generation copies. 

15 In such an application, there are three states that must be 

represented "copy never" , "copy once" and "copy no more" . The 
"copy never" state is intended to prohibit any copying of the 
content. The "copy once" state is intended to permit a single 
generation of copying. As such, this one generation copy must 

20 have its state changed from the original "copy once" to the 
"copy no more" state. 

The copy generation management can be implemented in a variety 
of ways. One solution is to use two different watermarks to 

25 encode the states "copy never" and "copy once" in the original 
material. Then, when a copy is made, an additional watermark 
is inserted that indicates the state "copy no more" . 
Several variations of this method are possible. First, the 
"copy once" watermark may be removed prior to the insertion of 

30 the "copy no more" watermark. In practice, this is probably 
unnecessary since more than one watermark can co- exist in the 
data. Also, if the watermark is easily removable, then it may 
be straightforward to circumvent the copy generation 
management system by simply removing the "copy once" watermark 

35 from the content. Content with no watermark is assumed to be 
freely copiable. 
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Second, the "copy once" state may be represented by the 
existence of two watermarks. One watermark would be difficult 
to remove and, when detected in isolation, would represent the 
"copy no more" state. The second watermark would be easy to 
5 remove and would only have meaning when detected in 

conjunction with the "copy no more" watermark, detection of 
both watermarks would represent the "copy once" state. An 
advantage of this approach is that, if carefully designed, the 
fragile watermark would be automatically removed by the 
10 existing installed base of VHS tape recorders. Note, however, 
that the fragile watermark must still survive signal 
processing that is common to video post production and 
transmission, e.g. low pass filtering, color correction and 
MPEG " 2 compr es s ion . 

15 

Watermarks are typically inserted as noise in the digital 
signal during the production stage. The insertion of the 
watermark can be accomplished by many different procedures 
known in the art. The watermark, although inserted as noise, 
20 must be invisible to human perception or the image quality 

will suffer. Thus watermarks inserted at the production stage 
are image dependent and require great computational effort to 
ensure that the watermark will be invisible but robust enough 
so that it will survive post signal processing. 

25 

Once inserted, it is necessary to detect the presence of the 
watermark at a subsequent time. A typical method for 
determining the presence of a watermark in suspect data is to 
compare some characteristic of the data or signal derived from 
30 the data characteristics to a target watermark pattern. The 
watermark is determined to be present if this comparison 
indicates sufficient similarity. 



BNSDOCID: <WO ^9963505A1_I_> 



wo 99/63505 



PCT/US99/08216 



-3- 



A typical method of the prior art for watermarking data is 
illustrated in FIG. 1. FIG. 1 shows three similar data sets 
and, above each, the signal that would be derived from the 
data characteristics during detection. To the original data 1 

5 a watermark pattern 2 is added. The sum of these two signals 
is the watermarked data 3. During the detection process a 
signal derived from data characteristics 6 is extracted from 
the watermarked data 3 and compared to a target watermark 
pattern 5. The watermarked data 3 is the sum of the original 

10 data 1 and the watermark pattern 2 . This may be a 

predetermined linear or non- linear combination. The signal 
derived from the data characteristics 6, in this example is a 
linear (or perhaps non- linear) combination of a signal derived 
from the original data 4 and the target watermark pattern 5 . 

15 In general, the characteristics of the watermark and those of 
the original data will interact and thus the signal derived 
from the characteristics of the watermarked data 6 and that 
derived from the characteristics of the watermark 5 will not 
exactly match. The similarity of the signal derived from the 

20 characteristics of the watermarked data 6 and the signal 

derived from the characteristics of the watermark 5 indicates 
the likelihood that the data contains the watermark 2. Note 
that in this illustration, the signal derived from the 
characteristics of the watermarked data 6 is similar to the 

25 signal derived from the characteristics of the watermark 5 

suggesting the watermark 2 is present in the watermarked data 
3, but the signal derived from the original data 4 and the 
signal derived from the characteristics of the watermarked 
data 6 are dissimilar, indicating that the original data 1 

30 does not yet contain the watermark 2 . 

Note that in FIG. 1, it is desirable for the watermarking 
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process to generate watermarked data 3 such that the extracted 
signal 6 is maximally similar to the target watermark 5. In 
many cases this similarity is increased as the relative 
amplitude or strength of the watermark pattern 2, and thus the 
5 signal extracted from the data characteristics 5, is 

increased. The drawback to increasing the watermark strength 
is that this also tends to decrease the similarity between the 
watermarked and original data 3, 1, respectively. Generally, 
in the case where the data represents audio, video, or 
10 imagery, this similarity is judged by the human visual and 
auditory systems. Thus, increasing the watermark strength 
often leads to a loss of audio, video, or image quality. 

A typical approach of the prior art to addressing these 
15 apparently contradictory concerns is to modify the watermark 
pattern such that when inserted at high enough strength for 
detection it introduces minimal perceptual degradation. 

In view of the prior art, there is a need for a method for 
20 inserting a watermark into digitized data for copy generation 
management which can be done at the recorder stage and thus 
requires minimal computational effort and associated costs. 
There is also a need for a method which inserts a watermark of 
a significantly lower strength then previously used in the 
25 prior art that can be detected with high likelihood. 

Therefore the present invention is directed to a method for 
data preparation and watermark insertion which requires a 
minimal computational effort at the recorder stage. 

30 

The present invention is further directed to a method for data 
preparation such that subsequent watermark insertion is 
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inexpensive as compared to the methods of the prior art. 

Still further, the present invention is directed to a method 
for data preparation and watermark insertion which results in 
5 a watermark of a significantly lower strength, then previously 
done in the prior art, which can be detected with high 
likelihood. 

The present invention is further directed to a method for data 
10 preparation and watermark insertion for copy generation 
management which addresses all of the above objectives. 

The method of the present invention differs from the typical 
approach in that not only is the watermark pattern carefully 

15 chosen to satisfy the tradeoff between high likelihood of 

detection and high perceptual similarity between watermarked 
and original data, but the method also modifies the 
characteristics and signals derived from the characteristics 
of the original data to reduce the interaction between the 

20 data characteristics and the watermark characteristics. This 
allows for a watermark of a significantly lower strength to be 
detected with high likelihood. 

To achieve these advantages over the prior art, a method for 
25 data preparation and watermark insertion is provided. The 
method comprises the step of preparing the data at a first 
time by manipulating at least one set of the data 
characteristics for subsequent insertion of a first watermark. 
In a preferred embodiment of the method of the present 
30 invention the method further comprises the step of inserting 
the first watermark by manipulating the set of data 
characteristics at a second time subsequent to the first time. 
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In still yet another preferred embodiment of the method of the 
present invention, the method further comprises the step of 
inserting a second watermark at a third time, before, during, 
or after the first time, by manipulating at least one set of 
5 the data characteristics. 

In a variation of the present invention a method for inserting 
a watermark into compressed data is provided. The compressed 
data has sets of data characteristics. The method comprises 
10 the steps of inserting a watermark by manipulating the set of 
data characteristics; and optimizing the manipulated data by 
modifying the compressed data characteristics subject to a set 
of constraints. 

15 These and other features, aspects, and advantages of the 

methods of the present invention will become better understood 
with regard to the following description, appended claims, and 
accompanying drawings where: 

20 FIG. 1 illustrates a typical method of watermark insertion of 
the prior art. 

FIG. 2 illustrates the method of watermark preparation and 
insertion of the present invention. 

25 
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FIG. 3 illustrates a flow chart generally outlining the steps 
of the present invention in the context of compressed data 
which iterates over all intracoded blocks. 

5 FIG. 4 illustrates a flow chart specifically outlining the 
steps of the present invention in the context of compressed 
data for each intracoded block iterated over all contributing 
coefficients . 

10 Although this invention is applicable to numerous and various 
types of digitized data, it has been found particularly useful 
in the environment of digital video data and copy generation 
management. Therefore, without limiting the applicability of 
the invention to digital video data and copy generation 

15 management, the invention will be described in such 
environment. 

The technique presented here is intended for an application in 
which two copy protection watermarks are inserted into image, 

20 audio, or video data. Prior to distribution, when ample 

computing resources may be available, the data is preprocessed 
for insertion of a watermark which will be referred to as the 
first watermark. While the data is preprocessed or prepared 
for insertion, the first watermark is not actually inserted at 

25 this time. Also prior to distribution, a second watermark may 
be added to the data. The second watermark is used to 
indicate to a consumer recording device such as digital 
versatile disk (DVD) , digital video home system (VHS) , digital 
audio tape (DAT) or recordable compact disk (CD-R) that no or 

30 one copies are allowed to be made. In the event that no 

copies are allowed, the recording device will prohibit such 
copies. In the event that one copy is allowed, the recording 
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device will allow copying, but will insert the first watermark 
into the copy. In the event that the first watermark is 
already present, further copying is disallowed. The method of 
the present invention encompasses the preparing of the data 
5 for the first watermark, along with the preferable steps of, 
inserting the first watermark, inserting a second watermark, 
and inserting the first watermark specifically into compressed 
data . 

10 Before describing the method of the present invention in 

detail the following overview of the three preferred steps of 
the present invention are discussed. In a preferred 
embodiment of the method of the present invention, the method 
comprises a first step of preparing the data at a first time 

15 by manipulating at least one set of the data characteristics 
for subsequent insertion of a first watermark, a second step 
of inserting the first watermark by manipulating the set of 
data characteristics at a second time subsequent to the first 
time, and a third step of inserting a second watermark at a 

20 third time, before, during, or after the first time, by 

manipulating at least one set of the data characteristics. 
These steps will now be discussed in detail below. However, 
it should be understood by someone skilled in the art that the 
first step alone is novel as compared to the prior art and 

25 that its combination with the next two steps is given by way 
of example only and not to limit the scope of the invention. 

Preparation 

30 A novel feature of the method of the present invention is the 
preparation of the data to be watermarked. The preparation 
allows for a watermark of low strength to be subsequently 
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inserted such that the watermark can be detected with high 
likelihood. 

Referring now to FIG. 2, there is illustrated the method of 
5 the present invention in which the original data is prepared 
for subsequent insertion of the first watermark. Instead of 
the watermark pattern 2, referred to as the first watermark, 
being added directly to the original data 1 as is done in the 
prior art, the watermark pattern 2 is added to a modified 
10 version 7 of the original data 1. The modification process 

that generates the modified version 7 from the original data 1 
is called the preparation stage. The two important properties 
of the modified data set 7 are that it is perceptually similar 
to the original data 1 and the data characteristics or signal 
15 derived from the data characteristics 8 is maximally different 
from that of the watermark data 5. 

In preferred applications of this invention, the original data 
may be audio, video, or still imagery. The set of data 

20 characteristics considered may be derived from the data in the 
spatial domain, the temporal domain, and/or a transformed 
domain. There are many transformed domains from which data 
characteristics can be derived. In the preferred 
implementation the data characteristics are derived from the 

25 data in both the spatial and the block DCT domains. Other 
local transform domains such as block Fourier transform, 
Hadamard transform, cortex transform, and wavelets as well as 
global transform domains such as the DCT and Fourier transform 
may be used. Spatial and temporal domain characteristic that 

30 can be used include sample value, edge features, color 
characteristics, textures, and phonemes. 
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The signals shown in FIG. 2, namely the signal derived from 
the characteristics of the original data 4, the signal derived 
from the characteristics of the_watermark 5, the signal 
derived from the characteristics of the watermarked data 6, 
and the signal derived from the characteristics of the 
prepared original 8 are derived from the data characteristics. 
The data characteristics are mapped to some signal having 
measurable features that will serve as the basis of the 
comparison in the detection process. The modified version of 
the data 7, after preparation, will have a derived signal 8 
that matches some criterion. In the preferred implementation, 
the criterion is minimum variance of the derived signal. In 
other words, we seek to make the derived signal as flat as 
possible. In general the data is modified such that the 
derived signal matches a target signal. Without loss of 
generality, this can be viewed as minimizing an error 
function, E, defined by equation 1: 

E = S - T (1) 

where S is the derived signal 8, and T is the target signal. 
In the preferred implementation we can consider the target 
signal, T to be a constant. 

25 Insertion of First Watermark 

Once the data has been prepared as described above, the first 
watermark can be added. Again, referring to FIG. 2, the 
preparation step has modified the unwatermarked data such that 
30 a signal 8 derived from a set of data characteristics has a 
known form. Using an error function as described above to 
shape the data characteristics, the derived signal 8 is 



20 



BNSDOCID: <WO 996350SA1J_> 



wo 99/63505 



PCT/US99/08216 



-11- 



preferably flat, as discussed previously, with average value 
close to zero. The watermark pattern 2 is designed such that 
the signal 5 derived from the same set of characteristics, 
this time extracted from the watermark pattern, is 
5 significantly different and easily distinguishable from the 

flat, zero mean signal 8 derived from the prepared data. The 
amplitude or strength of the watermark pattern 2 relative to 
the prepared data 7 may be very small while the signal derived 
from the watermarked data 6 will be sufficiently similar to 
10 the signal derived from the watermark pattern 5 to indicate a 
positive detection decision. 

Insertion of the first watermark involves a modification of 
the unwatermarked data such that a set of data 

15 characteristics, or a signal derived from a set of data 

characteristics, will closely match a known watermark signal, 
alternately referred to as a signal derived from a set of 
characteristics of a watermark pattern. The set of data 
characteristics considered may be derived from the data in the 

20 spatial domain, the temporal domain, and/or a transformed 

domain- There are many transformed domains from which data 
characteristics can be derived. In the preferred embodiment, 
the data characteristics are derived from the data in both the 
spatial and the block DCT domains. Other local transform 

25 domains such as block Fourier transform, Hadamard transform, 
cortex transform, and wavelets as well as global transform 
domains such as the DCT and Fourier transform may be used. 
Spatial and temporal domain characteristic that can be used 
include sample value, edge features, color characteristics, 

30 textures, and phonemes. In the spatial, temporal, and 

transformed domains the two step insertion process can reduce, 
to the addition of a fixed pattern to the data. This reduction 
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in complexity occurs because the data dependent, adaptive 
computations which are involved in typical watermark insertion 
(dependence on data characteristics discussed above) have been 
incorporated in the preparation stage. In the preferred 
embodiment the data characteristic used is an average of many 
block_DCT coefficients from various locations in the data. 

The insertion step may also be performed in a compressed 
domain. Examples of common video compression techniques in 
which this method is applicable are MPEG, MEPG2 , H.261, and 
H.263. JPEG is a good example of a still image compression 
technique in which the watermark preparation and insertion 
method of the present invention is applicable. In general, 
these compression techniques involve block DCT transformation 
of images or image frames (intracoding) or difference images 
(interceding) . in the later case the differences are 
typically between two video fram.es adjacent in time where one 
or both may be original frames or approximations to original 
frames. These compression techniques also contain a final, 
lossless entropy coding step. In general, the compressed data 
must be entropy decoded prior to watermark insertion. Some 
compression techniques allow for compressed frames in which 
both intra and inter coded blocks are present. The first 
watermark described is preferably inserted only into the 
25 intracoded blocks. 

Referring now to FIG. 3, there is illustrated a method of 
insertion of a first watermark into compressed data, the 
method generally referred to by reference numeral 300. The 
30 insertion into compressed data is a process in which each 

intracoded block is examined and modified. This modification- 
is made to optimize the. similarity of the watermark signal and 



20 
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the signal derived from the characteristics of the watermarked 
data under certain constraints, considering all changes made 
so far, and assuming no further changes will be made. Three 
important constraints are used. A fidelity constraint limits 

5 the maximum perceptual difference between the watermarked data 
and the prepared data. A bitrate constraint insures that the 
size of the compressed watermarked data does not differ from 
the compressed unwatermarked data. A level constraint 
restricts new watermarked values to be such that they will not 

10 be changed by subsequent quantization. Such quantization is 
usually required prior to reintroduction of the entropy 
coding . 

In summary, method 3 00 comprises the steps of analyzing each 
15 block of the processed data at step 302. When all blocks have 
been analyzed, the method 300 proceeds along route 302a to 
step 304 where it terminates. If there exists any blocks not 
yet analyzed, the method 300 proceeds along route 302b to step 
306 where a signal is derived from a set of data 
20 characteristics. At step 3 08 the method 300 determines the 

set of all possible changes that satisfy the constraints. At 
step 310 the change that would optimize the similarity between 
the derived signal and the target watermark in the absence of 
any future changes is selected. The method 300 then loops 
25 back to step 302 until all blocks have been analyzed. 

It should be understood that the method as illustrated in FIG. 
3 is given by way of example only, and not to limit the scope 
of the invention. 

30 

Insertion of Second Watermark 
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The second watermark can be inserted at any time relative to 
the preparation or to insertion of the first watermark. The 
combination of the second watermark and the preparation for 
the first watermark allow for powerful applications. 
5 Typically the set of data characteristics chosen for the 

second watermark should be disjoint from that chosen for the 
first watermark to avoid interaction between the two 
watermarks or interaction between the preparation process (for 
the first watermark) and the second watermark. However non- 
10 disjoint sets can also be supported. 

EXAMPLES 

Copy Generation Management : 

15 

In a preferred embodiment of the present invention in which 
the watermarks are used in a copy generation management 
scheme, it is preferable to add an additional watermark at the 
time of recording rather than to remove a watermark. The 

20 additional watermark is the first watermark discussed 

previously, which is computationally easy to insert and thus 
low cost hardware and/or software is capable of doing so, such 
as consumer DVD and VHS recorders. The first watermark also 
must not degrade the image fidelity and, at the same time, 

25 must be robust to subsequent signal transforms, e.g. survive 
digital - to -analog and analog- to-digital conversion. 

Insertion of the "copy once" watermark, referred to previously 
as the second watermark is computationally expensive since 
30 significant analysis of the image is performed to determine 

the perceptually important regions of the image. Thus, such- 
an insertion procedure is not practical for the first 
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watermark, yet the first watermark should have many of the 
characteristics of the more robust "copy once", second 
watermark. In this example, much of the computation 
associated with the first watermark is performed offline 
5 during the preparation stage, typically during production, 
such that a lightweight computational procedure can then be 
employed at the recorder i.e., the player stage, in order to 
insert the first watermark. 

10 Specifically, the second watermark is inserted into a set of 
averaged frequencies in the image. During or after insertion 
of the second watermark, a disjoint set of average frequencies 
are flattened or prepared for the subsequent insertion of the 
first watermark. 

15 

This preparation uses the same procedure as the second 
watermark, i.e. sophisticated perceptual models may be 
employed to best determine how the averaged frequencies can be 
flattened. This procedure prepares the image for the 
20 subsequent insertion of the first watermark. 

Insertion of the first watermark adds a unique watermark that 
represents the "copy -no -more" state. The strength of the 
first watermark- is significantly less than that commonly used 
25 for the second watermark. This is because the first watermark 
does not need to override the "noise" present in the image. 
This noise, a function of the naturally occurring signal that 
is the image, has been eliminated during the pre-processing 
phase where it is set to zero. 

30 

The first watermark has the same basic structure as other 
watermarks, e.g., it is preferably a N-bit binary vector In 
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the preferred embodiment N=64 however other values of N may be 
preferred to obtain higher data rates or lower computational 
costs . 

5 Insertion of the First Watermark into a MPEG- 2 Bitstream: 

In a preferred example of the method of the present invention, 
an algorithm for inserting the first watermark directly into 
the DCT coefficients of an MPEG- 2 stream is provided. The 

10 first watermark is inserted into intracoded blocks of an MPEG- 
2 layer. This may be the base layer of the MPEG- 2 stream or 
an enhancement layer. This method can be applied to an MPEG-1 
data stream as it qualifies as a valid MPEG- 2 base layer. 
This watermark insertion method affects neither the overall 

15 bitrate nor the starting positions of the slices in the MPEG- 2 
stream. The detection process is the same as that for the 
first watermark that had been inserted directly into the 
baseband video and the same as that for the second watermark. 
Detection processes for detecting the presence of a watermark 

20 in digital data are well known in the art. 

The goals of this technique are to insert the first watermark 
(increase the likelihood that it will be detected) , maintain 
the overall bitrate and position of I- frames within the data 
25 stream, and control the strength of the first watermark for 
both detection and visibility. 
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The method can be explained as follows. The 8x8 intracoded 
blocks of an MPEG stream are stored in the discrete cosine 
transform (DCT) domain. The location of the block within the 
image and the location of each coefficient within the 8x8 

5 block together specify to which, if any, watermark element 

that coefficient contributes. It is possible to predict the 
effect of an increase in a coefficient value on the 
correlation between the extracted and target watermarks. This 
effect may again be dependent on both the coefficient location 

10 within the 8x8 block and the block position within the image. 

It may also be dependent on the type of coding associated with 
the macroblock of which the current block is a member (field 
or frame based DCT) . Alternatively, it is possible to predict 
which change, positive, negative, or none, will increase the 

15 correlation between the extracted and target watermarks. 
Based on this prediction, the image can be examined on a 
block-by-block, basis and make coefficient changes which will 
increase the expected correlation. It is to be assumed that 
an increase in correlation will, in general, increase the 

20 correlation coefficient and thus the detection certainty. 

This method maintains the bitrate by insuring that the 
position in the data stream of the start of each slice or 
group of macroblocks does not change. This is done by 

25 requiring that any changes made, use the same or fewer bits to 
encode. If fewer bits are used and the slice becomes shorter, 
bits are padded onto the end of the slice to restore the 
starting position of the next slice in the data stream. These 
extra bits can instead be used in the slice to allow 

30 coefficient changes that use more bits to encode. We track 
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these extra bits or saved bits, increasing the count when a 
change decreases the required encoding bits and decreasing the 
count when a change increases the required encoding bits. The 
count of saved bits is not allowed to drop below zero. Thus, 
5 a change which increases the number of bits required for 

encoding is allowed only if there are enough bits saved from 
previous changes. At the end of the slice, the count of saved 
bits is reset to zero and those bits are padded into the data 
stream. 

10 

More specifically, MPEG stores the 63 AC coefficients of an 
intracoded block as run/value pairs, one pair for each non- 
zero coefficient. The value element of the pair refers to the 
signed magnitude of a non-zero coefficient. The run element 

15 refers to the number of zero valued coefficients directly 

preceding that non-zero coefficient. Each run/value pair is 
encoded with a unique binary code, called a variable length 
code (VLC) . In order to maintain the bitrate, only non-zero 
coefficients are considered and possible changes are evaluated 

20 based on the lengths of the original VLC and of that which 
would be necessary to encode the modified run/value pair. 

In addition to controlling the bitrate in the data stream, we 
wish to control the first watermark strength for detection and 

25 visibility- A stronger first watermark will be more 

detectable (increased likelihood of detection) but is also 
more likely to be visible. Within each block, the watermark 
strength is controlled for detection by a single value. This 
is the maximum amount of change allowed in the block. Once 

30 this maximum amount of change has been effected, no further 
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changes are allowed. 

The change within a block is distributed among the non-zero 
coefficients according to a frequency-based perceptual model. 

5 This is to control the visibility of the inserted first 

watermark, A frequency -based perceptual model is used to 
determine the relative insensi tivi ty of the human visual 
system (HVS) to the 63 AC DCT coefficients in an 8x8 block. 
These relative insensitivities are referred to as slacks. 

10 These slacks are dependent on the type of coding associated 
with the macrpblock of which the current block is a member 
(field or frame based DCT) . 

Referring now to Figure 4 there is illustrated a flow chart of 

15 the insertion method, referred to generally by reference 
numeral 400. MaxBlockChange refers to the strength for 
detection control as previously described. NumContrib is the 
number of non-zero AC coefficients for which a positive or 
negative change will increase the detected correlation. These 

20 coefficients are said to contribute to the correlation, 

TotalSlack is initialized as the sum of the slacks of all 
coefficients which contribute to the correlation at step 402. 
The method 400 loops over all contributing coefficients at 
steps 404 through 412 starting at the highest in the MPEG zig- 

25 zag scan order. The maximum change allowed for any 

coefficient is calculated at step 404 as the RemainingChange 
for the block weighted by the slack of that coefficient 
relative to the total remaining slack. The actual change is 
then found within the constraint of MaxChange and the constant 

30 bitrate constraint previously described. The function 
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ConstBitRate effects this change and updates the SavedBits 
count at step 406. The TotalSlack is then updated at step 408 
to represent the sum of the remaining slacks and the 
RemainingChange is updated, also at step 408, to represent the 
5 maximum amount the remaining contributing coefficients in the 
block may change. After all the contributing coefficients in 
the MPEG zig-zag scan order are processed the method 4 00 
terminates at step 414. 



10 While there has been shown and described what is considered to 
be preferred embodiments of the invention, it will, of course, 
be understood that various modifications and changes in form 
or detail could readily be made without departing from the' 
spirit of the invention. It is therefore intended that the 

15 invention be not limited to the exact forms described and 
illustrated, but should be constructed to cover all 
modifications that may fall within the scope of the appended 
claims . 
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WHAT IS CLAIMED IS ; 

1 1. A method for data preparation and watermark insertion, 

2 the data having sets of data characteristics, the method 

3 comprising the step of preparing the data at a first time 

4 by manipulating at least one set of the data 

5 characteristics for subsequent insertion of a first 

6 watermark . 

1 2. The method of claim 1, further comprising the step of 

2 inserting the first watermark by manipulating the set of 

3 data characteristics at a second time subsequent to the 

4 first time. 

1 3. The method of claim 1, further comprising the step of 

2 inserting a second watermark at a third time, before, 

3 during, or after the first time, by manipulating at least 

4 one set of the data characteristics. 

1 4. The method of claim 1, wherein the data is a 

1 representation of an image, audio or video. 

1 5. The method of claim 1, wherein the set of data 

2 characteristics are spatial and/or time domain 

3 characteristics . 

1 6. The method of claim 5, wherein the spatial and/or time 

2 domain characteristics are chosen from a group consisting 

3 of pixel intensities, edge features, color 

4 characteristics and textures. 
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1 7. The method of claim 1, wherein the set of data 

2 characteristics are transformed domain characteristics . 

1 8. The method of claim 1 , wherein the transformed domain 

2 characteristics are chosen from a group consisting of 

3 DCT, block DCT, FFT, block FFT, wavelet, Hadamard 

4 transform and Cortex transform. 

1 9. The method of claim 1, wherein the manipulating of the 

2 data characteristics is such that a signal is derived 

3 therefrom which substantially matches a pre- selected 

4 signal. 

1 10. The method of claim 9, wherein the pre -selected signal 

2 has zero variance. 

1 11. The method of claim 1, wherein the manipulating of the 

2 data characteristics is such that an error function is 

3 minimized. 

1 12. The method of claim 11, wherein the error function is a 

2 measure of variance. 

1 13. The method of claim 9, wherein the derived signal is 

2 performed by averaging subsets of the sets of data 

3 characteristics . 

1 14. The method of claim 2, wherein the inserting step occurs 

2 in a spatial and/or time domain. 

1 15. The method of claim 14, wherein the spatial and/or time ■ 

2 domain characteristics are chosen from a group consisting 
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3 of pixel intensities, edge features, color 

4 characteristics and textures. 

1 16. The method of claim 2, wherein the inserting step occurs 

2 in a transformed domain. 

1 17. The method of claim 16, wherein the transformed domain 

2 characteristics are chosen from a group consisting of 

3 DCT, block DCT, FFT, block FFT, wavelet, Hadamard 

4 transform and Cortex transform. 

1 18. The method of claim 2, wherein the inserting step occurs 

2 in a compressed domain resulting in compressed data. 

1 19. The method of claim 18, wherein the compressed domain is 

2 chosen from a group consisting of MPEGl , MPEG2 , JPEG, 

3 Quicktime, H.261 and H.263. 

1 20. The method of claim 18, wherein the compressed data is 

2 video and the first watermark is inserted into only 

3 intracoded portions of the compressed video data. 

1 21. The method of claim 18, wherein the further manipulation 

2 includes an optimization procedure that modifies the 

3 compressed data characteristics subject to a set of 

4 constraints. 

1 22. The method of claim 21, wherein the optimization 

2 procedure comprises the substeps of: 

3 (i) obtaining a set of values from the compressed data; 

4 (ii) determining a set of changes to the data that 

5 satisfy the set of constraints; 
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6 (iii) choosing a change from the set of changes that 

7 optimize detection of the watermark in the absence of any 

8 future changes; and 

9 (iv) repeating steps (i) through (iii) until all the 
10 compressed data is processed. 

1 23. The method of claim 21 or 22, wherein the set of 

1 constraints comprises fidelity constraints or bitrate 

2 constraints. 

1 24. The method of claim 21 or 22, wherein the set of 

2 constraints comprises constraining each value in the 

3 compressed data to a known quantization level. 

1 25. The method of claim 3, wherein the set of data 

2 characteristics manipulated for the second watermark are 

3 disjoint from the manipulated set of data characteristics 

4 for the first watermark. 

1 26. The method of claim 25, wherein the set of data 

2 characteristics manipulated for the second watermark and 

3 the set of data characteristics manipulated for the first 

4 watermark are disjoint in a spatial domain and/or a 

5 temporal domain and/or a transformed domain. 

1 27. The method of claim 2, wherein the first time occurs 

2 prior to data distribution and wherein the first 

3 watermark is inserted into the data at the time the data 

4 is copied. 

1 28. The method of claim 27, further comprising the step of 

2 prohibiting the copying of the data if the first 
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3 watermark is present. 

1 29. The method of claim 28, wherein the prohibiting step is 

2 implemented in a consumer recording device. 

1 30. The method of claim 29, wherein the format of the 

2 recording device is chosen from a group consisting of 

3 DVD, DHVS, DAT, and CD-R. 

1 31. A method for inserting a watermark into compressed data, 

2 the compressed data having sets of data characteristics, 

3 the method comprising the steps of: 

4 inserting a watermark by manipulating the set of data 

5 characteristics; and 

6 optimizing the manipulated data by modifying the 

7 compressed data characteristics subject to a set of 

8 constraints. 

1 32. The method of claim 31, wherein the optimization 

2 procedure comprises the substeps of: 

3 (i) obtaining a set of values from the compressed data; 

4 (ii) determining a set of changes to the compressed data 

5 that satisfy the set of constraints; 

6 (iii) choosing a change from the set of changes that 

7 optimize detection of the watermark in the absence of any 

8 future changes; and 

9 (iv) repeating steps (i) through (iii) until all the 
10 compressed data is processed. 

1 33. The method of claim .31 or 32, wherein the set of 

1 constraints comprises fidelity constraints or bitrate 

2 constraints. 
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1 34. The method of claim 31 or 32, wherein the set of 

2 constraints comprises constraining each value in the 

3 compressed data to a known quantization level. 
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